orrign  Affairs 
Representatives 


Q> 

United  States 

General  Accounting  Office 

Washington,  D.C.  20548 


Program  Evaluation  and 
Methodology  Division 

B-2 11376 

August  11,  1989 

The  Honorable  Dante  B.  Fascell 
Chairman,  Committee  on  Foreign  Affairs 
House  of  Representatives 

Dear  Mr.  Chairman: 

This  report  »-an  unclassified  version  of  our  report  on  recent  develop¬ 
ments  in  the  Bigeye  weapon  program  entitled  Bigeye  Bomb:  Unresolved 
Developmental  IssuesJ-&ftO/e4*EMB-89-i).  The  classified  report  contains 
eight  appendixes  of  detailed  data  and  analysis  that  we  are  unable  to 
discuss  in  this  unclassified  report. 

The  Senate  Appropriations  Committee  report  on  the  fiscal  year  1987 
Department  of  Defense  (dod)  appropriations  bill  requested  that  dod  sub¬ 
mit  a  report  on  four  “outstanding  Bigeye  concerns”  involving  (1)  prob¬ 
lems  of  excessive  pressure  buildup,  (2)  the  generation  of  lethal  agent. 

(3)  agent  “flashing”  (or  burning),  and  (4)  the  overall  reliability  of  the 
weapon.  In  December  1986.  your  staff  asked  us  to  review  the  dod  report 
when  it  became  available  and  to  report  to  you  on  it,  as  well  as  on  other 
related  Bigeye  bomb  issues. 

This  report  analyzes  the  September  1987  Department  of  Defense  report 
(composed  of  a  letter  and  seven  appendixes),  which  was  issued  in 
response  to  the  fiscal  year  1987  Senate  Appropriations  Committee 
requirements,  as  well  as  other  relevant  dod  reports  and  data.1  Five  of 
the  seven  appendixes  in  the  dod  report  were  prepared  by  the  U.S. 

Army’s  Chemical  Research,  Development,  and  Engineering  Center 
(crdec)  in  Aberdeen,  Maryland.  A  sixth,  on  vx  flashing,  was  written  by  a 
former  crdec  official  who  is  now  a  consultant.  The  seventh  and  final 
appendix,  a  table  on  Bigeye  system  reliability,  was  issued  by  dod’s  Big¬ 
eye  program  office. 

In  preparing  our  analysis,  we  reviewed  the  September  1987  dod  report 
and  crdec  1988  data  that  were  included  with  the  August  1, 1988,  dod 
response  to  a  draft  of  our  report.  In  addition,  we  analyzed  findings  from 
a  June  1988  analysis  of  1987  Bigeye  operational  field  tests  by  the 


'The  DOD  report  is  composed  of  a  letter  from  the  Assistant  to  the  secretary  of  Defense  (Atomic 
Energy)  and  seven  appendixes  to  John  C.  Stennis,  Committee  on  Appropriations,  U.S.  Senate.  Wash¬ 
ington,  D.C.,  and  is  dated  September  10,  1987. 
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Navy’s  Norfolk,  Virginia,  operational  test  and  evaluation  force 
(optevfor)  that  are  relevant  to  Bigeye  developmental  problems. J 

We  also  interviewed  dod  officials  involved  in  the  Bigeye  program, 
including  crdec  personnel  and  officials  in  the  Bigeye  program  manager’s 
office.  We  utilized  the  expertise  of  outside  consultants  on  chemical  reac¬ 
tions  to  ensure  that  our  evaluation  had  a  sound  scientific  basis. 

In  response  to  a  request  from  the  dod  Inspector  General,  we  asked  a 
panel  of  four  experts  to  help  resolve  the  issues  outstanding  between  tod 
and  gao — which  are  discussed  in  the  present  report — concerning  the 
test  and  evaluation  of  the  Bigeye  bomb.  The  experts  were  Dr.  John 
Ahearne,  Vice  President,  Resources  for  the  Future;  Dr.  John  Gibbons, 
Director,  Office  of  Technology  Assessment,  U  S.  Congress;  Dr.  Joseph 
Navarro,  former  dod  Deputy  Undersecretary  (Test  and  Evaluation)  in 
the  office  of  the  Undersecretary  of  Defense,  Research  and  Engineering; 
and  Admiral  Ernest  R.  Seymour  (ret.),  former  commander  of  Navy  Air 
Systems  (navair).  In  addition  to  reading  our  reports  on  Bigeye  issues,  the 
panel  was  briefed  by  TOD  on  January  30,  1989,  with  regard  to  dod ’s 
response  to  the  draft  of  this  report  and  subsequently  reported  their 
findings  and  conclusions  to  the  Comptroller  General  .1 

The  panel’s  findings  were  summarized  in  a  March  30,  1989,  memoran¬ 
dum  that  was  sent  from  the  Comptroller  General  to  the  Secretary  of 
Defense. 1  The  panelists  concluded  that  we  were  correct  in  our  evaluation 
that  major  developmental  issues  affecting  the  Bigeye  remain  unresolved 
and  that  further  developmental  tests  are  therefore  required  to  answer 
questions  that  critically  affect  the  bomb’s  performance. 

We  delayed  this  report  so  that  the  Congress  could  have  the  benefit  of 
the  panel’s  judgment,  along  with  our  own  findings.  Our  analyses  of 
developmental  issues  in  each  of  the  four  relevant  areas  are  summarized 
in  the  next  four  sections  of  this  letter.  In  a  fifth  section,  we  discuss  our 
analysis  of  lethal-agent  presistency. 


-Commander  of  Operational  Test  and  Evaluation  Force,  “Multiservice  Operational  Evaluation  of  the 
Bigeye  BEU-80/B  Weapon"  (Norfolk,  Virginia:  June  24,  1988). 

‘Our  previous  reports  on  B’gpye  issues  are  Bigeye  Bomb-  An  Evaluation  of  POP's  Chemical  aim 
Developmental  Tests,  GAO/C-l’EMD-86-lBR  (May  1986)  and  Bigeye  Bomb:  1988  Status  Report,  GAO/ 
PEMD-88-26  (May  1988). 

'Utter  from  GAO  Comptroller  General  to  Secretary  of  Defense'  Richard  B.  Cheney,  dated  March  31. 
1989. 
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ckdkc  conducted  one  additional  test  (designated  LB-44)  to  determine 
pressure  buildup  in  an  unvented  bomb  with  a  high  initiation  tempera¬ 
ture  (140  F),  as  required  by  the  Congress.  On  the  basis  of  this  one  addi¬ 
tional  test,  dod  concluded  that  pressure  buildup  is  not  likely  to  be  a 
problem  for  the  Bigeye. 

However,  a  sample  of  one  additional  test  is  clearly  insufficient  to  deter¬ 
mine  whether  the  results  provide  a  reliable  estimate  of  the  true-but- 
unknown  success  rate  for  the  universe  of  pressure  tests.  Standard  sta¬ 
tistical  formulas  show  that  five  tests,  all  of  which  must  be  successful, 
are  required  to  assure  that  there  is  a  50-percent  chance  that  the  success 
rate  from  the  universe  of  tests  with  a  pass-fail  criterion  would  fall  some¬ 
where  between  80  and  100  percent. 

dod  officials  stated  at  the  final  (exit)  conference  on  this  report  that  the 
one  additional  pressure  test  may  not  be  sufficient  to  reach  a  sound  con¬ 
clusion  about  the  susceptibility  of  Bigeye  to  excessive  pressure  buildup 
that  could  destroy  or  disable  the  weapon. 

We  conclude  that  dod  did  not  respond  adequately  to  the  congressional 
concern  about  excessive  pressure  buildup  inside  the  bomb  and  that  the 
issue  of  pressure  buildup  remair  i  resolved. 


Purity  and  Lethality 


Change  in  Bigeye  TEMP 
Requirements 


In  its  September  1987  report,  dod  provided  no  explanation  (although 
required  to  do  so  by  the  Senate  Appropriations  Committee)  of  why  it 
changed  the  requirement  concerning  the  number  of  seconds  required  to 
generate  binary  vx  (the  lethal  agent  formed  inside  the  Bigeye  bomb)  at 
the  purity  or  equivalent-biotoxicity  level  required  by  the  Test  and  Eval¬ 
uation  Master  Plan  (temp).' 

The  second  (January  1987)  Bigeye  temp  stated  that  vx  at  the  temp- 
required  level  need  be  generated  only  “at  any  time”  (that  is,  at  some 
instant)  during  the  5  through  30-second  period  after  mixing  starts,  with¬ 
out  specifying  either  when  the  required  purity  or  equivalent  biotoxicity 
level  should  be  reached  or  for  how  long.  The  first  (May  1985)  Bigeye 


VX  is  0,S  ethyl  diisopropylaminoethyl  methyl  phosphonothioate,  a  persistent  nerve  agent,  Biotoxie- 
ity  is  a  method  of  measuring  the  lethality  of  the  generated  agent. 
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Results  of  Lethal-Agent 
Generation  Tests 


temp  did  not  contain  the  “at  any  time”  phrase,  dod  states  that  “at  any 
time”  is  a  clarification  of  the  meaning  of  the  first  temp  rather  than  a 
revision  of  the  purity  or  equivalent  biotoxicity  requirement. 

However,  we  believe  that  a  bomb  that  produces  lethal  agent  at  the 
required  purity  level  for  1  second  (or  even  less)  may  not  be  as  effective 
a  weapon  as  one  that  does  so  for  25  seconds.  Furthermore,  it  is  impor¬ 
tant  to  know  when  the  1-second-of-adequately-pure-agent  will  be  pro¬ 
duced  so  that  the  pilot  can  release  the  bomb  at  the  appropriate  time. 
Unfortunately,  it  is  not  known  when  the  adequate  purity  level  will  occur 
during  the  mixing  reaction. 

dod  has  apparently  revised  the  purity  criterion  in  another  respect,  since 
it  now  states  that  the  vx  requirement  is  for  a  purity  or  equivalent- 
biotoxicity  level  that  is  different  from  the  one  specified  by  the  temp.  We 
find  that  this  is  an  incorrect  understanding  by  dod  of  the  temp  require¬ 
ment  and  is  not  justified,  dod  disagrees  with  our  views  on  this  issue,  but 
further  details  cannot  be  discussed  because  the  exact  figures  are  classi¬ 
fied.  (See  appendix  IV  of  the  classified  version  of  this  report.) 


We  differ  with  dod  on  the  number  of  lethal-agent  generation  tests  con¬ 
ducted  by  crdec  in  which  vx  at  the  TEMP-required  purity  or  equivalent 
biotoxicity  level  was  produced  at  some  point  during  the  required  5 
through  30-second  period,  dod’s  evaluation  of  test  results  finds  that  vx 
of  sufficient  lethality  was  generated  much  more  often  than  our  evalua¬ 
tion  indicates,  dod  uses  its  estimate  of  lethal  agent  success  as  part  of  its 
calculation  of  weapon  system  reliability.  (See  the  following  sections  for 
further  discussion.) 

dod  states  that  the  evidence  for  its  evaluation  of  test  results  is  fully  doc- 
umentable  and  accurate.  However,  Bigeye  program  officials  acknowl¬ 
edged  in  July  1988  that  they  had  no  definitive  list  of  the  tests  used  by 
the  former  program  manager  to  arrive  at  an  estimate.  In  April  1989,  dod 
provided  us  with  a  revised  list  of  13  tests  that,  it  stated,  showed  that  the 
temp  requirement  had  been  met.  However,  dod  reached  that  conclusion 
by  using  the  incorrect  vx  criterion,  cited  previously,  and  by  declaring 
test  results  to  be  “no-tests”  in  some  cases  where  the  purity  requirement 
was  not  met. 
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Can  Biotoxicity  Be 
Predicted  From  VX  Purity 
Using  Available  Data? 


Conclusions:  Purity- 
Biotoxicity  Issues 


We  question  whether  vx  biotoxicity  can  be  reliably  predicted  from  vx 
purity,  as  stated  in  the  dod  September  1987  report  and  in  a  June  1988 
report  on  more  recent  tests.  This  is  an  important  issue  because  the  Big- 
eye  program  office  stated  in  July  1988  that  certain  tests  are  successes, 
in  terms  of  meeting  purity  equivalent-biotoxicity  requirements,  on  the 
basis  of  predicting  biotoxicity  from  observed  vx  purity  that  did  not 
reach  the  TKMP-required  vx  purity  criterion. 

However,  in  January  1988,  the  crdec  munitions  test  director  questioned 
the  reliability  of  the  equation  used  in  the  1987  report,  stating  that  there 
were  insufficient  data  on  which  to  base  the  equation.  More  than  2  years 
ago,  moreover,  in  our  May  1986  report  (gao/cpemd-86-ibr)  on  the  Bigeye 
program,  we  quoted  dod  officials  as  stating  that  on  the  basis  of  the  same 
data  “the  relationship  between  chemical  purity  and  biotoxicity  cannot 
be  considered  statistically  significant.”  Yet,  on  the  basis  of  these  data, 
dod  concluded  in  the  September  1987  report  that  the  bomb  will  produce 
vx  with  a  percutaneous  biotoxicity  that  exceeds  the  temp  requirement. 

crdec  issued  a  revised  purity  equivalent-biotoxicity  equation  in  a  June 
17,  1988,  report,  but  warned  that  the  equation  cannot  be  used  to  extra¬ 
polate  beyond  the  data  in  it.11  Even  with  more  data  from  two  new  1988 
tests,  there  are  still  just  14  biotoxicity  data  points,  from  six  tests.  Of 
these  tests,  only  one  had  a  high-temperature  start  of  140  "F,  none  had 
start  temperatures  between  71"F  and  129"F,  none  had  a  start  tempera¬ 
ture  over  140  "F,  and  two  OF  tests  produced  conflicting  results. 

The  June  1988  optevfor  analysis  of  Bigeye  operational  tests  noted  the 
“limited”  number  of  data  points  on  purity -biotoxicity  and  pointed  out 
that  of  the  14  data  points,  only  11  were  from  tests  using  the  current 
bomb  configuration  for  off-station  mixing,  optevfor  formally  recom¬ 
mended  additional  tests  with  start  temperatures  in  the  140-160  F  range, 
to  provide  needed  purity-biotoxicity  data.  We  believe  that  the  weight  of 
the  evidence  shows  that  the  currently  available  data  are  inadequate  to 
make  reliable  predictions  about  the  level  of  vx  purity  required  to  gener¬ 
ate  a  given  level  of  equivalent  biotoxicity. 


We  conclude  that  dod  has  not  dealt  adequately  with  the  Senate  Appro¬ 
priation  Committee’s  concern  about  achieving  adequate  lethality  in  the 
Bigeye  bomb.  We  find  that  dod’s  view  that  the  addition  of  the  phrase  “at 
any  time”  to  the  temp  is  only  a  clarification — not  a  change — in  the 


“CRDEC,  "Bigeye  Agent  Purity  and  Biotoxicity  Relationship,”  June  17,  1988. 
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requirement  understates  the  importance  of  the  modification.  It  does  not 
appear  to  accurately  reflect  the  implications  of  generating  adequately 
pure  agent  for  only  an  instant  during  the  period  from  5  to  30  seconds,  in 
contrast  ti  generating  it  throughout  the  25  seconds. 

Moreover,  dod  now  appears  to  have  loosened  the  criterion  in  another 
regard  by  stating  that  it  requires  vx  at  a  purity  or  equivalent-biotoxicity 
level  that  is  different  from  the  one  in  the  temp.  We  find  this  to  be  an 
incorrect  understanding  of  the  temp  requirement.  We  also  concur  with 
Ol’TEVPOR’s  recommendation  that  additional  tests  be  conducted  to  collect 
purity-biotoxicity  data. 


Flashing 


In  response  to  the  Senate  Appropriations  Committee's  requirement  to 
provide  test  results  on  the  level  of  resistance  of  vx  to  flashing  (burning 
or  vaporization  of  an  agent),  crdec  contracted  with  a  former  crdec  offi¬ 
cial  to  conduct  an  analysis  and  synthesis  of  related  literature.  The  dod 
report  concludes  that  flashing  caused  either  by  an  external  spark  or  by 
autoignition  is  very  unlikely  to  be  a  problem  in  Bigeye  deliveries. 

However,  we  believe  this  conclusion  may  be  hasty  since  we  think  there 
is  evidence  that  flashing  could  in  fact  occur  through  autoignition — burn¬ 
ing  or  vaporization  that  results  from  the  combination  of  a  heated  mate¬ 
rial  and  a  fuel-rich  oxygenated  atmosphere  that  is  not  ignited  by  an 
external  spark.  In  four  of  six  Bigeye  tests  with  temperature  starts  at  or 
over  130  ’F,  the  temperature  generated  inside  the  bomb  equaled  or 
exceeded  the  vx  autoignition  temperature  and,  in  the  fifth,  came  within 
3  C  of  it. 

dod  states  that  the  bomb’s  design  precludes  externally-ignited  flashing 
and  that  the  high  recovery  rate  of  simulants  used  in  operational  tests 
and  the  absence  of  observed  flashing  are  evidence  that  neither  exter¬ 
nally-ignited  nor  autoignition  flashing  is  likely  to  be  a  problem. 

However,  the  tests  cited  were  not  designed  as  tests  of  flashing  and  had 
no  instrumentation  by  which  to  record  internal  bomb  temperatures. 
Moreover,  some  of  the  tests  used  nonreactive  bis  simulant,  which  does 
not  generate  the  heat  or  pressure  typical  of  a  Bigeye  binary  reaction.7 

We  also  question  the  conclusions  drawn  on  the  basis  of  a  crdec  test  of 
the  bomb  cited  as  important  evidence  against  the  possibility  of  either 


*  BIS  is  BIS-(2  ethyl  hexyl)  hydrogen  phosphite.  It  is  used  as  a  nonreactivc  simulant  of  VX 
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externally-ignited  or  autoignition  flashing,  because  the  test  weapon  was 
surrounded  by  a  bonnet  of  inert  Halon,  a  fire-suppressant  chemical,  that 
the  dod  report  states  may  have  reduced  chances  for  flashing.  Further, 
the  reaction  generated  an  internal  maximum  temperature  of  only  223  "C, 
far  short  of  the  290C  autoignition  point  of  vx. 

Moreover,  in  August  1988,  the  munitions  director  of  crdec  noted  that  no 
data  currently  exist  on  the  autoignition  points  of  impure  vx  mixtures, 
which  are  the  product  of  the  binary  reaction  inside  the  Bigeye,  and  that 
it  would  therefore  be  a  good  idea  to  collect  such  data.  Without  this  infor¬ 
mation,  it  is  impossible  to  know  whether  the  internal  temperatures 
reached  in  tests  of  the  bomb  exceed  autoignition  points  of  the  generated 
(impure)  agent  and,  therefore,  whether  autoignition  flashing  could  have 
occurred. 

We  conclude  that  dod  has  not  adequately  addressed  the  issue  of  flashing 
and  that  additional  data  on  the  issue  need  to  be  developed. 


System  Reliability 


dod  agrees  with  us  that  its  September  1987  estimate  of  overall  Bigeye 
system  reliability  was  calculated  improperly  and  that  the  overall  relia¬ 
bility  estimate  is  therefore  wrong. 

Another  problem  is  dod’s  assumption  regarding  the  generation  of  lethal 
agent.  We  believe  that  a  more  accurate  estimate  of  operational  reliability 
must  be  based  on  the  actual  laboratory  test  success  rate  for  generating 
vx  at  the  required  level  of  purity  or  equivalent  biotoxicity.  Using  data 
from  these  tests,  we  determined  that  vx  at  the  TEMP-required  level  was 
produced  much  less  often  than  at  the  rate  estimated  by  dod. 

We  conclude,  therefore,  that  the  weapon  system  reliability  is  considera¬ 
bly  lower  than  dod’s  estimate. 


VX  Persistency 


The  dod  September  1987  report  states  that  vx  is  very  persistent,  based 
in  part  on  a  test  conducted  in  a  chamber  containing  0-percent  humidity, 
a  constant  temperature  of  73"F,  and  no  wind.  We  do  not  question  that  vx 
is  very  persistent  under  these  specific  conditions.  However,  we  believe 
that  this  test  failed  to  provide  operationally  relevant  information  about 
the  degrading  effects  on  vx  of  wind,  moisture,  soil,  and  other  absorbent 
factors. 
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This  issue  was  also  cited  by  the  Air  Force,  which  noted  in  an  annex  to 
the  optkvfor  report  that  “more  information  on  the  effects  of  delivering 
Bigeye  in  visible  moisture  such  as  clouds,  fog,  rain,  etc.  is  needed  to 
assess  the  advisability  of  continuing  a  mission  under  such  conditions.” 
We  concur  with  the  Air  Force’s  judgment  that  such  information  is  essen¬ 
tial  to  evaluating  the  effectiveness  of  Bigeye  and  must  be  collected. 


Summary 


We  conclude  that  dod  has  not  adequately  addressed  the  requirements  in 
the  fiscal  year  1987  Senate  Appropriations  Committee  report  and  that 
additional  information  or  tests  are  required  to  answer  basic  develop¬ 
mental  questions  about  the  performance  of  the  Bigeye  bomb  in  areas 
critical  to  weapon  performance:  pressure  buildup,  purity,  flashing,  and 
persistency.  Furthermore,  estimates  of  overall  weapon  reliability  must 
take  into  account  both  the  lethality  of  the  agent  generated  and  the  dura¬ 
tion  of  that  lethality. 

Moreover,  we  concur  with  optevfor’s  formal  recommendation  in  its  June 
1988  report  that  no  further  operational  tests  be  carried  out  until  a 
number  of  steps  are  taken  with  regard  to  the  weapon,  including  the  fol¬ 
lowing  that  involve  developmental  issues:  conducting  additional  purity 
and  biotoxicity  tests  at  start  temperatures  from  140-160"F,  researching 
chemical  additives  that  would  keep  start  temperatures  under  120"F, 
determining  the  increased  vx  deposition  levels  required  when  agent 
purity  is  below  the  TEMP-required  level,  and  improving  weapon  reliabil¬ 
ity.  We  believe  that  optevpor’s  recommendations  show  the  need  for 
more  information  on  these  developmental  issues  critical  to  Bigeye  effec¬ 
tiveness,  and  we  agree  that  further  operational  tests  should  be  delayed 
until  these  basic  developmental  issues  are  resolved. 

While  this  report  was  not  provided  to  dod  for  agency  review,  the  classi¬ 
fied  draft  upon  which  this  report  is  based  was  provided  to  dod  for  com¬ 
ment.  In  commenting  on  the  classified  report,  dod  disagreed  with  most  of 
the  major  concerns  we  raised.  However,  after  careful  review  of  dod’s 
comments  and  the  relevant  evidence,  and  after  reconsidering  the  find¬ 
ings  of  the  expert  panel,  we  are  confident  that  our  concerns  are  well- 
founded.  (dod’s  comments  and  our  response  to  them  can  be  found  in 
appendix  VIII  of  the  classified  report.) 

Our  review  was  conducted  in  accordance  with  generally  accepted  gov¬ 
ernment  auditing  standards. 
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If  you  have  any  questions,  please  call  me  or  Carl  E.  Wisler,  Director  of 
Planning  and  Reporting,  at  275-1854.  Major  contributors  to  this  report 
are  listed  in  appendix  I. 

Sincerely, 

Eleanor  Chelimsky 
Assistant  Comptroller  General 
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